Skip to content

dtak/wide-bnns-public

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

6 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Wide Mean-Field Bayesian Neural Networks Ignore the Data

This repository contains the code necessary to reproduce the experiments in our paper, Wide Mean-Field Bayesian Neural Networks Ignore the Data (AISTATS 2022).

Bayesian neural networks (BNNs) combine the expressive power of deep learning with the advantages of Bayesian formalism. In recent years, the analysis of wide, deep BNNs has provided theoretical insight into their priors and posteriors. However, we have no analogous insight into their posteriors under approximate inference. In this work, we show that mean-field variational inference entirely fails to model the data when the network width is large and the activation function is odd (e.g., tanh). We also show this need not be true if the activation function is not odd (e.g., ReLU).

figure1

Reproducibility

To reproduce our results you'll first need to train BNNs. For more instructions, open commands.txt in each of the following folders, each corresponding to one or more figures:

  • experiments/experiment_1/work: Figures 1 and 4
  • experiments/experiment_2/work: Figure 2
  • experiments/experiment_3/work: Figures 5 and 6

After training the models, run experiments/figures/work/make_figures.py.

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages